Universal Benchmark Suites
نویسنده
چکیده
We present concepts, design, and use of universal benchmark suites. Such suites consist of benchmark programs that represent all relevant characteristic types of workload. The execution times of individual benchmarks and appropriate weights can be used to compute global performance indicators that reflect a spectrum of specific compound workloads. All such global indicators can be obtained from the same universal benchmark suite. This approach substantially reduces the cost of benchmarking.
منابع مشابه
Quantitative Methods for Design of Benchmark Suites
In this paper we present black-box models of work-load diierence and use them to develop quantitative methods for the design of benchmark suites. The proposed design methods (the method of optimum subsets , and the method of optimum weights) combine a new concept of program space coverage functions, and the criteria of completeness, redundancy, density, and granularity of benchmark suites.
متن کاملSPEC HPG benchmarks for high-performance systems
In this paper, we discuss results and characteristics of the benchmark suites maintained by the Standard Performance Evaluation Corporation’s (SPEC) High-Performance Group (HPG). Currently, SPEC HPG has two lines of benchmark suites for measuring performance of large-scale systems: SPEC OMP and SPEC HPC2002. SPEC OMP uses the OpenMP API and includes benchmark suites intended for measuring perfo...
متن کاملOpenML Benchmarking Suites and the OpenML100
We advocate the use of curated, comprehensive benchmark suites of machine learning datasets, backed by standardized OpenML-based interfaces and complementary software toolkits written in Python, Java and R. Major distinguishing features of OpenML benchmark suites are (a) ease of use through standardized data formats, APIs, and existing client libraries; (b) machine-readable meta-information reg...
متن کاملThe pitfall in comparing benchmarks using hardware performance counters
Understanding the behavior of emerging workloads is important for designing next generation microprocessors. For addressing this issue, computer architects and performance analysts build benchmark suites of new application domains and compare the behavioral characteristics of these benchmark suites against well-known benchmark suites. Current practice typically compares workloads based on micro...
متن کامل